The instant invention generally relates to systems and methods for detecting targets from radar signals and more particularly to systems and methods utilizing neural network processing to detect targets from a Frequency Modulated Continuous Wave (FMCW) automotive radar.
A vehicle may contain automatic safety restraint actuators that are activated responsive to a vehicle crash for purposes of mitigating occupant injury. Examples of such automatic safety restraint actuators include air bags, seat belt pretensioners, and deployable knee bolsters. One objective of an automatic restraint system is to mitigate occupant injury, thereby not causing more injury with the automatic restraint system than would be caused by the crash had the automatic restraint system not been activated. Generally, it is desirable to only activate automatic safety restraint actuators when needed to mitigate injury because of the expense of replacing the associated components of the safety restraint system, and because of the potential for such activations to harm occupants. This is particularly true of air bag restraint systems, wherein occupants too close to the air bag at the time of deploymentxe2x80x94i.e. out-of-position occupantsxe2x80x94are vulnerable to injury or death from the deploying air bag even when the associated vehicle crash is relatively mild. Moreover, occupants who are of small stature, or with weak constitution, such as children, small adults or people with frail bones, are particularly vulnerable to injury induced by the air bag inflator. Furthermore, infants properly secured in a normally positioned rear facing infant seat (RFIS) in proximity to a front seat passenger-side air bag are also vulnerable to injury or death from the deploying air bag because of the close proximity of the infant seat""s rear surface to the air bag inflator module.
Air bag inflators are designed with a given restraint capacity, as for example, the capacity to protect an unbelted normally seated fiftieth percentile occupant when subjected to a 30 MPH barrier equivalent crash, which results in associated energy and power levels which can be injurious to out-of-position occupants. While relatively infrequent, cases of injury or death caused by air bag inflators in crashes for which the occupants would have otherwise survived relatively unharmed have provided the impetus to reduce or eliminate the potential for air bag inflators to injure the occupants which they are intended to protect.
Known deployment systems for vehicle safety devices such as an air bag require the host vehicle to actually collide with an obstacle or other vehicle before the deployment decision process begins. At that point in time, the sensors detect a deceleration in the host vehicle and deploy one or more safety systems. Thus, the crash is identified based solely on the characteristic of the acceleration versus time measure. The disadvantage with existing post-crash detection systems derives from the fact that the time available to deploy an active safety device is relatively short, particularly for side impact or high speed frontal collisions where occupant restraint systems can provide significant safety benefits. These short time frames lead to rates of inflation of the air bags that are so great that injury or death are possible if the occupant is not properly situated with respect to the air bag.
Ideally, the air bag would be inflated prior to any interaction with a normally seated occupant, and at a rate which is sufficiently slow that an out of position occupant would not be injured by the inflating air bag. For a crash of sufficient severity, this requires the crash sensing system to be able to predict immanent crashes because the time required to inflate the bag at an inflation rate sufficiently slow to be safe for out-of-position occupants may be greater than either that required for the occupant to move so as to commence interaction with an inflated air bag or to safely decelerate the occupant.
Typically, predictive collision sensing systems utilize radar to sense the range and possibly velocity of a target relative to the vehicle. A radar system measures the distance and/or velocity of a target by sensing the effects of the interaction of the target with a beam of wave either continuous or pulsed energy, whereby the range to the target is determined by measuring the transit time of the radar signal, and the velocity of the target is determined by measuring the Doppler frequency shift of the received backscattered signal relative to the transmitted signal.
Frequency Modulated Continuous Wave (FMCW) radar generally operates by illuminating one or more targets with a constant amplitude transmitted signal which is frequency modulated over time. In a Linear FMCW (LFMCW) radar, the frequency modulation is linear with respect to time, or in other words, the transmitted signal is chirpedxe2x80x94either up-chirped, down-chirped, or a combination of the two where the associated direction indicates the relative change in frequency with time. The transmitted signal is backscattered off each target back to an associated receiver, thereby undergoing a propagation delay which is proportional to the range of the target relative to the transmitter and receiver. For each point of reflection on each target, the corresponding received signal has a time varying frequency content similar to the transmitted waveform but shifted in time by a time delay proportional to the time required for the signal to travel from the transmitter to the target and then to the receiver. For a system with the transmitter and receiver relatively fixed with respect to one another, this time delay is also proportional to the range of the target thereto. Typically, the transmitter and receiver either share a common antenna, or utilize separate antennas which are in close proximity to one another. The received signal is recombined with either the transmitted signal, or a replica thereof as generated by a local oscillator, to produce a intermediate signal having a beat or intermediate frequency which is proportional to the target range. Alternately, the received signal may be separately combined with two different local oscillator signals, one of them being in-phase with the transmitted signal, the other being in phase quadrature thereto, which both effectively increases the sampling rate and enables a distinction with respect to the sign of the Doppler shift in the received backscattered signal in accordance with the associated target velocity. A sequence of received backscattered signals may be used to unambiguously resolve both target range and target velocity.
Some of the advantages of FMCW radar compared with other types of radar are 1) the modulation thereof is readily compatible with a wide range of solid-state transmitters, 2) the measurement of frequency necessary for range measurements can be performed digitally using a Fast Fourier Transform (FFT), and 3) the FMCW signals are relatively difficult to detect with conventional intercept receivers.
Traditional Linear Frequency Modulated (LFM) signal processing of a continuous wave (CW) radar signal requires a number of steps of processing including: 1) read the A/D converters and format the resulting data, 2) remove leakage, DC bias and in-phase/quadrature-phase imbalance in the received signal, 3) Fourier Transform processing, and 4) peak finding and detection and Constant False Alarm Rate (CFAR) detection.
This processing is inherently pipelined (serial) in time rather than parallel in time which leads to significant processing throughput requirements, depending on the LFM sweep update time. The only way to effectively parallelize the algorithm is to either context switch between tasks on a single processor that operates four times as fast as a single task requires or to have four dedicated processors, one for each task as seen in FIG. 1a. Alternately as illustrated in FIG. 1b, if a single time-slicing processor is used it requires four times the clock speed needed to execute a single task, and therefore is more expensive and requires more power.
The problem is further complicated since the signal experiences a significant amplitude gain through the FFT processing (Order(N2) growth in amplitude), which requires the processing hardware to have large data word sizes to prevent overflow and to avoid the need for on-the-fly data scaling which can effect the final signal to noise ratio of the system. This is highly inefficient since the FFT amplitude values can require up to 32 bits to encode with no clipping but the final target detection values are limited to the maximum range of the system which is 50-100 meters and is encodable in 6-7 bits.
Generally in a continuous wave radar system, a leakage signal is always present together with the signal backscattered by targets. Moreover, the leakage signal often has a much higher amplitude than the target signal, so that subtracting the leakage can be an important step in the signal processing of CW radar in order to extract a useful target signal. A Kalman filter may be employed to track the leakage amplitude and deduce the amount to be subtracted from each sample to account for the leakage.
For systems requiring multiple antenna beams, a leakage signal for each beam would typically be stored in computer memory, which can be relatively memory intensive. The leakage signal results from a component of the transmitted signal which leaks into the receiver without first interacting with the target. The amount of signal leakage is generally greater for systems where the transmit and receive circuits share a single antenna, than for systems having distinct transmit and receive antennas. This leakage signal is generally collected through a dedicated calibration process, where the antenna is aimed at the sky and data is collected for each beam. This calibration may also be temperature dependent depending on the materials use to develop the antenna array, so that even more leakage signals would need be collected and stored in order to compensate for this effect.
As a signal is always mixed with noise, a group of potential target ranges generally appears after all the above processing. To obtain the ranges of true targets and maintain a stable false alarm rate, the detection process employs an order statistics based approach for the constant false alarm rate detection.
The principal disadvantages of prior art FMCW signal processing methods include the following 1) high computer hardware throughput requirements, 2) large computer hardware data word sizes, 3) large amounts of memory, 4) excessive power requirements (due to high performance processor chips), 5) large amounts of software due to all the aspects of the algorithm, and 6) leakage signal and DC bias signals can mask small targets unless specifically removed from processing.
Neural networks are known for providing a general and robust framework for the approximation of a variety of functions. For example, it is known that a neural network can approximate an FFT function, particularly for FMCW radar signal processing. Neural networks are also known for solving classification or discrimination problems, for example determining whether a point represented by a particular input vector is inside or outside of a particular region.
A variety of neural networks of various topologies are known, including the perceptron, the multilayer perceptron, the MADALINE, radial basis function networks, associative memories, Jordan/Elman networks, Hopfield networks, principal component analysis networks, Kohonen self-organizing maps (SOFM), adaptive resonance theory (ART) networks, Fukushima Neocognitron networks, and time lagged recurrent networks. In general, neural networks comprise an interconnected set of processing elements, wherein each processing element comprises at least one weighted input and an output, wherein the value of the output is given by a generally non-linear transformation of the linear combination of the weighted inputs. The values of the weights for each processing element are established through a training process, for example backpropagation, wherein a set of inputs are provided, the resulting output response of the neural network is compared with the desired output, and the values of the weights for one or more inputs to one or more processing elements are adjusted so that the output response is changed towards the desired response.
The perceptron comprises a feedforward neural network with no hidden processing elements, that is capable of discriminating amongst linearly separable classes. For example, in two dimensions with inputs x1 and x2, a processing element having an output y that implements the threshold function y=U(x1xe2x88x92x2), where U is the unit step function, is able to classify points as to their location (above or below) relative to the line x1=x2.
The multi-layer perceptron (MLP) comprises one or more hidden layers of processing elements between the input and output layers, wherein the outputs from the input layer are inputs to the first hidden layer, the outputs from the last hidden layer are inputs to the output layer, and the outputs from one hidden layer are inputs to the next corresponding hidden layer. For static classification, the MLP with two hidden layers is a universal pattern classifier, wherein the discriminant functions can take any shape.
The Hopfield network is a recurrent neural network with no hidden nodes and with symmetric weights, wherein the processing elements comprise adders followed by a threshold nonlinearity, whereby the network is stable in the sense of Lyapunov so that the state of the system approaches the nearest equilibrium point for a given set of inputs. With fixed weights corresponding to pre-selected equilibrium points, the Hopfield network acts as an associative memory, with some of the same features as human memory.
Known neural network systems either do not involve FMCW collision radar, or do not teach or suggest systems or methods for solving the problems peculiar to FMCW collision radar, such as the removal of leakage and DC bias from the detected backscattered radar return signal.
The instant invention overcomes the above-noted problems by providing a system and method of processing a radar signal using a neural network that processes the intermediate frequency in-phase and quadrature phase signals sampled in time from a FMCW radar to detect the range of targets illuminated by the associated radar transmit signal, wherein the result of processing by the neural network is similar to the result from the conventional FMCW signal processing steps of DC bias removal, leakage removal, Fast Fourier Transformation, and CFAR detection. The associated neural network radar processor can be implemented on a neural network processor chip for reduced cost and improved reliability. The instant invention also provides a method of training the neural network with signals that are germane to an FMCW radar, including a wide range of target scenarios as well as leakage signals, DC bias signals, and background clutter signals.
In accordance with a first aspect, a neural network radar processor comprises a multilayer perceptron neural network comprising an input layer, a second layer, and at least a third layer, wherein each layer has a plurality of nodes. Each node of the input layer is operatively connected to every node of the second layer. The second and third layers comprise a plurality of subsets of nodes, wherein nodes from one subset of the third layer are operatively connected only to nodes of one subset of the second layer, there being a one-to-one correspondence between subsets in the second and third layers. The respective interconnected subsets constitute respective mutually exclusive subnetworks. The outputs of the nodes in the third layer are operatively connected to outputs of the neural network, and the neural network is trained so that each output node is responsive to a target at in particular range cell and each subnetwork is responsive to targets within a set of target ranges, wherein different subnetworks are responsive to respectively different non-overlapping sets of target ranges.
In accordance with a second aspect, a method of processing a radar signal comprises forming a time series by time sampling in-phase and quadrature-phase components of a down-converted radar return signal, forming a first plurality of first weighted sums, and forming a second plurality of second weighted sums. Each component of the down-converted radar return signal comprises an associated waveform and the time series comprises a plurality of elements. Each first weighted sum comprises a sum of first values weighted by a corresponding set of first weights, and each first value is a first function of a different element of the time series. Each second weighted sum comprises a sum of second values weighted by a corresponding set of second weights. The first plurality of first weighted sums comprises a third plurality of mutually exclusive subsets of the first weighted sums and the second plurality of second weighted sums comprises a fourth plurality of mutually exclusive subsets of the second weighted sums. Each second value of a second weighted sum of one of the fourth plurality of mutually exclusive subsets is a second function of a different first weighted sum of only one of the third plurality of mutually exclusive subsets of first weighted sums. The method further comprises adjusting the first and second sets of weights so that the third set of weighted sums of one of the fourth plurality of mutually exclusive subsets is substantially responsive only to targets in a first set of target ranges, and the third set of weighted sums of another of the fourth plurality of mutually exclusive subsets is substantially responsive only to targets in a second set of target ranges, whereby the first and second sets of target ranges are not overlapping.
In accordance with a third aspect, a method of training a neural network in a neural network radar processor comprises forming at least one first time series of in-phase and quadrature-phase components representing a down-converted radar return signal from a target space and applying the at least one first time series to an input layer of a neural network. Each component comprises an associated waveform and the at least one first time series comprises a plurality of elements. The input layer comprises a first plurality of first nodes. One of the elements of the at least one first time series is applied to each of the first plurality of first nodes, and different elements are applied to different first nodes. The neural network further comprises a second plurality of nodes, a third plurality of corresponding weight values operative at the second plurality of nodes, and a fourth plurality of output nodes. The method further comprises adjusting the third plurality of corresponding weight values so that an output signal at each of the fourth plurality of output nodes is representative of the target space. The at least one first time series is selected from a time series representative of a target space, a time series with a leakage signal and a time series with a DC bias; the target space is selected from a target space with a variation of target reflection size, a target space with target fluctuation, a target space with a variation of target position relative to the radar, a target space with a variation in target speed relative to the radar, a target space with a variation in a target viewing angle, a target space with a variation in target shape, a target space with a variation in target size, a target space with a variation in antenna beam pointing angle, a target space with background clutter, and a target space with glint. For the at least one first time series selected from a time series with a leakage signal but no target, a time series with a DC bias but no target, and a time series representing a target space with background clutter, the third plurality of corresponding weight values are adjusted so that the output signal at each of the fourth plurality of output nodes tend towards a nullity.
These and other objects, features, and advantages of the instant invention will be more fully understood after reading the following detailed description of the preferred embodiment with reference to the accompanying drawings. While this description illustrates the application of the instant invention in predictive crash sensing system using LFMCW radar, it will be understood by one with ordinary skill in the art that the instant invention can also be applied to general FMCW radar signal processing for identifying the range and velocity of one or more targets.